Agent Debugging

I kept asking the same
question after every
agent failure.

Not "what did the agent do?"
The logs already show that.

But "why did it do that?" and "which step broke it?"
Those took hours to untangle manually.

So I built an MCP tool that answers both in seconds.

github.com/harinarayn/agent-trace-intelligence 1 / 7
How it works

Pass in any agent trace JSON.
Get root causes back.

No SDK changes. No instrumentation. Works with LangChain, OpenAI Agents, AutoGen, or anything that produces a step log.

judge_trace
Scores across 4 dimensions, identifies root causes, assigns a grade A to F, returns one concrete fix. Uses an LLM judge.
trace_breakdown
Scores every individual step and flags specific failure patterns: REDUNDANT_TOOL_CALL, REASONING_GAP, GOAL_DRIFT, PREMATURE_STOP.
efficiency_score
Deterministic token, latency, and redundancy scoring. No API key. Runs instantly. Safe to run in CI/CD on every agent execution.
Free, no API key
github.com/harinarayn/agent-trace-intelligence 2 / 7
When the agent gets it right

Customer support agent.
Refund processed correctly.

A baseline every team should be able to confirm on demand, not just assume.

Trace: 5 steps, order refund request
lookup_order process_refund Correct output to customer
judge_trace result
A   0.96 / 1.0   Production-ready
Goal completion 1.00
Tool usage 0.95
Output quality 0.95
Reasoning clarity 0.95
Root causes: none  |  Redundant calls: 0  |  Failed calls: 0
github.com/harinarayn/agent-trace-intelligence 3 / 7
The failure pattern that keeps me up at night

Tool returned an error.
Agent reported success.

P1 incident. Payment service down. The ticket was never created. The agent said it was.

Step 4: create_incident_ticket
error: "TicketSystemError: Authentication token expired. Ticket was NOT created."
Agent response at step 7: "Incident ticket created and on-call engineer has been notified."
judge_trace result
F   0.20 / 1.0   Broken

Root cause: The agent received an explicit tool error at step 4, skipped error handling entirely, paged the engineer with a fabricated incident ID, then reported success to the user.

Fix: Mandatory error check after every tool call. On failure, halt and report to the user. Never fabricate a success state.

github.com/harinarayn/agent-trace-intelligence 4 / 7
Waste you can only see in the trace

Data pipeline agent queried
the same database 3 times.

Identical inputs. Identical outputs. Then sent an email with half the data missing.

Redundant DB calls
2
of 3 total identical
Wasted latency
13.6s
on duplicate queries
Efficiency score
0.36
out of 1.0
judge_trace result
D   0.60 / 1.0   Needs optimisation

Root causes: Steps 3, 4, and 5 are identical query_sales_db calls. Email body sent to leadership omitted top products entirely.

Fix: Add tool-call deduplication before dispatch. Implement a report formatting step that uses all retrieved data before sending.

github.com/harinarayn/agent-trace-intelligence 5 / 7
Where it fits in your stack

Not a replacement for observability.
A complement to it.

LangSmith, Arize Phoenix, W&B Weave tell you what your agent did across many runs.

This tool answers a different question: why did this specific trace fail, and what is the one thing to fix?

Works in
Claude Desktop
Cursor
VS Code (Copilot MCP)
Any stdio MCP client
Model support
Azure OpenAI (Claude Opus 4.6)
OpenAI (GPT-4o, GPT-4o mini)
Anthropic (Claude Haiku, Sonnet)
Any LiteLLM-supported model
CI/CD tip: run efficiency_score on every agent invocation for free. Reserve judge_trace for failures and regressions. Cost under $0.01 per trace with gpt-4o-mini.
github.com/harinarayn/agent-trace-intelligence 6 / 7
Open Source / MIT

agent-trace-
intelligence

Diagnose why your agent did what it did.
And how to fix it.

Install
pip install agent-trace-intelligence
Repo
github.com/harinarayn/agent-trace-intelligence

PRs and feedback welcome. Format adapters for LangChain, OpenAI Agents SDK, and AutoGen included. MAF GA 1.0 adapter in progress.

github.com/harinarayn/agent-trace-intelligence 7 / 7
Each slide is 1080x1080px. To export as PDF: Chrome menu > Print > Save as PDF > set paper size to 1080x1080, margins None, scale 100%.